Control of a physical system based on inferred state

ABSTRACT

A system and computer-implemented method are provided for enabling control of a physical system based on a state of the physical system which is inferred from sensor data. The system and method may iteratively infer the state by, in an iteration, obtaining an initial inference of the state using a mathematical model representing a prior knowledge-based modelling of the state, and by applying a learned model to the initial inference of the state and the sensor measurement, wherein the learned model has been learned to minimize an error between initial inferences provided by the mathematical model and a ground truth and to provide a correction value as output for correcting the initial inference of the state of the mathematical model. Output data may be provided to an output device to enable control of the physical system based on the inferred state.

CROSS REFERENCE

The present application claims the benefit under 35 U.S.C. § 119 ofEuropean Patent Application No. EP 19161069.0 filed on Mar. 6, 2019,which is expressly incorporated herein by reference in its entirety.

FIELD

The present invention relates to a system and computer-implementedmethod for enabling control of a physical system based on a state of thephysical system which is inferred from sensor data. The presentinvention further relates to a computer-readable medium comprising datarepresenting instructions arranged to cause a processor system toperform the computer-implemented method.

BACKGROUND INFORMATION

Conventionally, a physical system may be controlled based on a state ofthe physical system which is determined from sensor data. For example,the movement of an autonomous agent, such as a robot, car or drone, maybe controlled based on a current location of the agent, with the currentlocation being determined from location data obtained from a GlobalPositioning System (GPS) sensor or similar type of geolocation sensor.However, such location data does not represent the ‘true’ location ofthe agent but rather a sensor-based measurement, which may deviate fromthe true location due to uncertainty in the measurements, which is alsosimply referred to as ‘noise’ and which may be due to various reasons,e.g., in this example due to satellite signal blockage, multipathinterference, radio interference, etc.

It is however desirable to be able to determine the true location of theagent as accurately as possible based on such potentially noisy sensordata. Similar examples of where it is desirable to determine a state ofa physical system based on potentially noisy sensor data include heatingsystems for buildings (being an example of a physical system), in whichit may be desirable to determine the temperature (being an example of astate) using sensor data obtained from a temperature sensor, or anelectric motor (being another example of a physical system), in which itmay be desirable to determine the rotor's position (being anotherexample of a state) based on sensor data obtained from a positionfeedback sensor.

It will be appreciated that sensor measurements may be associated with astate, but typically represent measurements of an observable quantitywhile the state itself may be an unobservable quantity, e.g., forfundamental or practical reasons. For example, the temperature istypically measured indirectly, e.g., using a temperature sensor whichmeasures changes in resistance, with these changes in resistancecorrelating to variations in temperature. Other examples of the sensormeasurement only indirectly relating to a particular state of a physicalsystem include, but are not limited, to the following: the orientation(being an example of a state) of a device may be determined from anaccelerometer which measures so-called ‘proper’ acceleration, and theoccupancy of a room (being another example of a state) may be determinedfrom infrared measurements obtained in the room.

Conventionally, sensor data may be filtered to reduce noise in thesensor data, thereby allowing the state to be more accurately determinedfrom such sensor data. For example, Kalman filtering may be used torecursively estimate a state of a physical system based on a time-seriesof sensor measurements.

Such Kalman filtering may incorporate prior knowledge about the relationbetween the sensor measurements and the state of the physical system.For example, a mathematical model may be used which represents a priorknowledge-based modelling of the state as a function of a sensormeasurement and a previous inferred state. Such a mathematical model mayfor example incorporate the physical laws of motion to estimate acurrent location of a robot based on a previous location of the robotand geolocation measurements obtained from a GPS sensor in the robot.

SUMMARY

It may be desirable to enable control of a physical system based on astate of the physical system which is inferred from sensor data, inwhich the inference of the state is improved over Kalman filtering andsimilar techniques.

In accordance with a first aspect of the present invention, an examplesystem for enabling control of a physical system is provided. Inaccordance with a further aspect of the present invention, an examplecomputer-implemented method is provided for enabling control of aphysical system. In accordance with a further aspect of the presentinvention, an example computer-readable medium is provided and anexample computer-readable medium is provided.

The above measures provide an iterative inference of a state of aphysical system based on sensor data which represents sensormeasurements associated with the state of the physical system. Thephysical system may for example be a physical entity such as a vehicle,robot, etc., or a connected or distributed system of physical entities,e.g., a lighting system, or any other type of physical system, such as abuilding. The state may be time-varying quantity, e.g., a temperature, aspeed, an acceleration, a position, a geolocation, an occupancy, etc.,but also a set of such quantities, e.g., a vector of speed andacceleration.

The state may be inferred from sensor data which may be obtained fromone or more sensors. The sensor(s) may be part of the physical system soas to be able to obtain sensor measurements which are associated withthe state of the physical system, or may be arranged separate from thephysical system if the sensor measurements may be obtained remotely. Thesensor measurements pertain to quantities that allow the state to beinferred from the sensor measurements. As such, there exists acorrelation between the measured quantity(s) and the quantity(s)representing the state to be inferred. Typically, the quantityrepresenting the state to be inferred cannot be measured directly, e.g.,for fundamental or practical reasons.

The above measures involve iteratively inferring the state of thephysical system. For that purpose, in an iteration of the inference, asensor measurement is obtained, after which an initial inference of thestate is obtained using a mathematical model representing a priorknowledge-based modelling of the state. More specifically, the sensormeasurement and the previous inferred state are used as input to themathematical model, which then produces the initial inference of thestate as output. The mathematical model may encode prior knowledge onthe relation between the sensor measurement and the state to beinferred, with the model taking into account the previous inferredstate. For example, the mathematical model may represent a physicalmodelling of the relation between the sensor measurement and the stateto be inferred. In a specific example, if the state is a temperature tobe measured and the quantity which is measured is the resistance of amaterial, the mathematical model may express the temperature as afunction of the measured resistance and a previous inferred temperature.In generating the mathematical model, domain knowledge may be used, suchas a temperature coefficient of resistance (TCR) of the material. Ingeneral, the mathematical model may be constructed at least in partbased on manual specification, e.g., by a system designer, and expressedas one or a set of equations. Such types of mathematical models may beknown per se from the fields of statistics and control systems.

The above measures further involve providing and applying a learnedmodel to the initial inference of the state and the sensor measurement.The learned model, which may represent any suitable learned model suchas a neural network, has been learned to minimize an error betweeninitial inferences provided by the mathematical model and a groundtruth. As output, a correction value is provided for correcting theinitial inference of the state of the mathematical model. The state ofthe physical system to which the sensor measurement pertains is thenobtained by combining the initial inference of the state provided by themathematical model and the correction value provided by the learnedmodel, for example by applying the correction value to the initialinference of the state by means of a simple addition.

The above measures have the effect that a mathematical model is used forobtaining an initial estimate of the state of the physical system, whichis then corrected by a learned model which has been specifically learnedto correct the initial inference of the mathematical model based on aground truth.

Effectively, the prior knowledge-based modelling by the mathematicalmodel may provide a coarse estimate of the state of the physical system,while the learned model may provide a refinement of the coarse estimate.Namely, the learning on the ground truth may have enabled the learnedmodel to recognize unknown relations or correlations between the sensormeasurement and the state to be inferred which have not been representedin the prior knowledge-based modelling by mathematical model.

Surprisingly, such a hybrid approach of using a prior knowledge-based(‘non-learned’) mathematical model and a learned model has been found toimprove the accuracy of the inference of the state not only compared tothe use of a prior knowledge-based modelling alone, but also compared tothe use of a learned model which has been learned to directly infer thestate of the physical system based on the sensor measurement. Namely,such ‘directly’ learned models may struggle to accurately model thedynamics of a physical system, e.g., due to an insufficient complexityof the learned model and/or due to insufficient availability of trainingdata. Conversely, the learned model used in the above measures may beless complex and/or require less training data as the relation(s)between the sensor measurements and the state may have been alreadymodelled to an approximate degree by the mathematical model. As such,the training may be limited to having to learn the typically smallerdeviations in the initial inference from the ground truth.

Having inferred the state of the physical system based on the sensormeasurements, output data may be provided to an output device which isused in the control of the physical system so as to enable the controlof the physical system based on the inferred state. For example, theoutput device may be an actuator which is part of, or located in avicinity of the physical system. The output data may be used to controlthe actuator, and thereby the physical system. In other examples, theoutput device may be a rendering device which is configured to generatea sensory perceptible output signal based on the inferred state. Forexample, the rendering device may be a display configured to display theoutput signal. Other types of sensory perceptible output signalsinclude, but are not limited to, light signals, auditive signals,tactile signals, etc. Corresponding rendering devices are known per se.Based on the sensory perceptible output signal, an operator may thencontrol the physical system. For example, the sensory perceptible outputsignal may denote a failure in a component of the physical system, whichmay prompt an operator to control the physical system by stopping orpausing its operation. In general, the sensory perceptible output signalmay represent the inferred state, or may represent a result derived fromthe inferred state, e.g., a failure diagnosis.

Optionally, the processor subsystem is configured to iteratively inferthe state of the physical system by using time-series of sensormeasurements and previous inferred states. Effectively, instead of usingonly a previous sensor measurement and a previous inferred state, arespective time-series may be used as input to the mathematical modeland the learned model. Each time-series may be limited in temporalextent, and may be represented by a sliding window which may, e.g., beimplemented as a circular buffer. The use of such time-series mayprovide a more accurate inference of the state as the dynamics of thephysical system may be better approximated by the mathematical modeland/or such an approximation may be more accurately refined by thelearned model.

Optionally, the learned model is a recurrent neural network (RNN), andthe processor subsystem is configured to maintain and pass a hiddenstate of the recurrent neural network between iterations of theiterative inference of the state of the physical system. Such recurrentneural networks are well-suited as learned model when iterativelyinferring the state of a physical system as such types of neuralnetworks exhibit temporal dynamic behaviour, and therefore may be usedto learn refinements to the approximation of a state of a dynamicphysical system. For example, the learned model may be a graph neuralnetwork (GNN) comprising a gated recurrent unit (GRU) to establishrecursion in the graph neural network.

Optionally, the mathematical model comprises a transitional model partwhich models a conditional probability of the state to be inferred givena previous inferred state, and a measurement model part which models aconditional probability of the sensor measurement given the state to beinferred. The mathematical model may thus be a probability-based modelwhich models the state as at least two conditional probabilities: afirst ‘transitional’ probability representing the conditionalprobability that the previous inferred state transitions to a currentinferred state, and a second ‘measurement’ probability representing theconditional probability that the sensor measurement denotes a currentinferred state. Such type of prior knowledge-based modelling iswell-suited to be processed by mathematical estimation techniques suchas Kalman Filtering, for example when assuming that probabilitydistributions of the mathematical model are each linear and Gaussian.

It will be appreciated by those skilled in the art that two or more ofthe above-mentioned embodiments, implementations, and/or optionalaspects of the present invention may be combined in any way deemeduseful.

Modifications and variations of the computer-implemented method or anycomputer-readable medium, which correspond to the describedmodifications and variations of the system, can be carried out by aperson skilled in the art on the basis of the present description, andvice versa.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects of the invention will be apparent from andelucidated further with reference to the embodiments described by way ofexample in the description below with reference to the figures.

FIG. 1 shows a system for enabling control of a physical system, whereinthe system is configured to iteratively infer a state of the physicalsystem from sensor data obtained from a sensor and to provide outputdata to an output device associated with the control of the physicalsystem, such as an actuator.

FIG. 2 illustrates a Hidden Markov Process.

FIG. 3 illustrates an iteration of the iterative inference in anembodiment in which the inference is modelled as a message passingscheme.

FIG. 4 shows a method for enabling control of a physical system.

FIG. 5 shows a computer-readable medium comprising data.

It should be noted that the figures are purely diagrammatic and notdrawn to scale. In the figures, elements which correspond to elementsalready described may have the same reference numerals.

LIST OF REFERENCE NUMBERS

The following list of reference numbers is provided for facilitatingunderstanding of the figures and shall not be construed as limiting thepresent invention.

-   20 sensor-   40 output device-   60 physical system-   100 system for enabling control of physical system-   120 input interface-   122 sensor data-   140 output interface-   142 output data-   160 processor subsystem-   180 data storage interface-   190 data storage-   192 data representation of mathematical model-   194 data representation of learned model-   200 method for enabling control of physical system-   210 accessing sensor data-   220, 222 iteration of iterative inference-   230 obtaining sensor measurement-   240 obtaining initial inference from mathematical model-   250 obtaining correction value from learned model-   260 combining initial inference with correction value-   270 providing output data to output device-   300 computer-readable medium-   310 non-transitory data

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

The following relates to a system and computer-implemented method forenabling control of a physical system based on a state of the physicalsystem which is inferred from sensor data. Specific examples of suchphysical systems, states to be inferred and types of sensor data havebeen indicated in the summary section. The following elaborates onvarious aspects of the implementation of the system andcomputer-implemented method, and on the iterative inference itself.

FIG. 1 shows a system 100 for enabling control of a physical systembased on a state of the physical system which is inferred from sensordata. FIG. 1 shows the physical system schematically as dashed outline60, but in general, such a physical system may for example be a physicalentity such as a vehicle, robot, etc., or a connected or distributedsystem of physical entities, e.g., a lighting system, or any other typeof physical system, e.g., a building. The physical system 60 is shown tocomprise a sensor 20 which may measure one or more quantities whichcorrelate with a state of the physical system which is to be inferred.As shown in FIG. 1 , the sensor 20 may be part of the physical system.In other examples, the sensor 20 may be arranged remotely from thephysical system 60, for example if the quantity(s) can be measuredremotely. For example, a camera-based sensor may be arranged outside ofa robot but may nevertheless measure quantities associated with therobot, such as the robot's position and orientation within a workspace.

The system 100 is shown to comprise an input interface 120 which isshown to access sensor data 122 from the sensor 20. Effectively, theinput interface 120 may represent a sensor interface. Alternatively, theinput interface 120 may access the sensor data from elsewhere, e.g.,from a data storage or a network location. Accordingly, the inputinterface 120 may have any suitable form, including but not limited to alow-level communication interface, e.g., based on I2C or SPI datacommunication, but also a data storage interface such as a memoryinterface or a persistent storage interface, or a personal, local orwide area network interface such as a Bluetooth, Zigbee or Wi-Fiinterface or an ethernet or fibreoptic interface.

The system 100 is further shown to comprise an output interface 140 toprovide output data 142 to an output device 40, which may for example bean actuator 40 which is part of the physical system 60. For example, theactuator may be an electric, hydraulic, pneumatic, thermal, magneticand/or mechanical actuator. Specific yet non-limiting examples includeelectrical motors, electroactive polymers, hydraulic cylinders,piezoelectric actuators, pneumatic actuators, servomechanisms,solenoids, stepper motors, etc., etc. In another example, which is notshown in FIG. 1 , the output device may be a rendering device, such as adisplay, a light source, a loudspeaker, a vibration motor, etc., etc,which may be used to generate a sensory perceptible output signal whichmay render the inferred state or a result derived from the inferredstate, such as a failure diagnosis or any other type of derived result,e.g., for use in guidance, navigation or other type of control of thephysical system.

The system 100 is further shown to comprise a processor subsystem 160configured to iteratively infer the state of the physical system basedon the sensor data 122 accessed via the input interface 120, and toprovide output data 142 to the output device 40 to enable the control ofthe physical system 60 based on the inferred state. For that purpose,the processor subsystem 160 may be configured to, in an iteration of theinference, obtain a sensor measurement, obtain an initial inference ofthe state using a mathematical model representing a priorknowledge-based modelling of the state as a function of the sensormeasurement and a previous inferred state, apply a learned model to theinitial inference of the state and the sensor measurement, wherein thelearned model has been learned to minimize an error between initialinferences provided by the mathematical model and a ground truth and toprovide a correction value as output for correcting the initialinference of the state of the mathematical model, and obtain a currentinferred state by combining the initial inference of the state with thecorrection value.

The system 100 is further shown to comprise a data storage interface 180for accessing a data storage 190, which may be a volatile ornon-volatile type of data storage, and which may be used to temporarilyor persistently store data used by the processor subsystem 160,including but not limited to a data representation of the mathematicalmodel 192 and a data representation of the learned model 194.

Various details and aspects of the operation of the system 100,including optional aspects, will be further elucidated with reference toFIGS. 2 and 3 .

In general, the system may be embodied as, or in, a single device orapparatus, such as a workstation or a server. The server may be anembedded server. The device or apparatus may comprise one or moremicroprocessors which execute appropriate software. For example, theprocessor subsystem may be embodied by a single Central Processing Unit(CPU), but also by a combination or system of such CPUs and/or othertypes of processing units. The software may have been downloaded and/orstored in a corresponding memory, e.g., a volatile memory such as RAM ora non-volatile memory such as Flash. Alternatively, the processorsubsystem of the system may be implemented in the device or apparatus inthe form of programmable logic, e.g., as a Field-Programmable Gate Array(FPGA). In general, each functional unit of the system may beimplemented in the form of a circuit. The system may also be implementedin a distributed manner, e.g., involving different devices orapparatuses, such as distributed local or cloud-based servers. In someembodiments, the system may be part of the physical system itself,and/or may be represent a control system configured to control thephysical system.

Various embodiments are possible for iteratively inferring the state ofa physical system based on sensor data, while using in the iterativeinference a ‘hybrid’ approach involving a prior knowledge-based(‘non-learned’) mathematical model and a learned model. The followingexample describes the iterative inference by modelling the iterativeinference as a directed graphical model which uses an iterative messagepassing scheme over edges of the directed graphical model. It will beappreciated, however, that such a message passing scheme may serve toexplain the iterative inference, but the actual implementation of theiterative inference may be carried out in various other ways, e.g., onthe basis of analogous mathematical concepts. In particular, what is inthe following referred to as ‘prior knowledge messages’ may represent amessage passing-based representation of the mathematical model, whilethe ‘learned messages’ may represent a message passing-basedrepresentation of the learned model, as also discussed elsewhere.

Furthermore, in following example, a recurrent neural network is used aslearned model, being in this example a graph neural network (GNN)comprising a gated recurrent unit (GRU) to establish recursion in thegraph neural network. It will be appreciated, however, that other typesof recurrent neural networks may be used as well, or in general anyother type of learned model providing the functionality as described.The inference itself is based on the mathematical model comprising atransitional model part which models a conditional probability of thestate to be inferred given a previous inferred state, and a measurementmodel part which models a conditional probability of the sensormeasurement given the state to be inferred, and the probabilitydistributions of the mathematical model being assumed to be linear andGaussian. As such, the following may be considered a ‘hybrid’ approachto Kalman Filtering by incorporating a learned model as described.

As indicated above, the iterative inference may be modelled as adirected probabilistic graphical model (henceforth also simply referredto as ‘generative model’) using a message passing scheme where the nodesof the graphical model can send and receive messages to infer estimatesof the states x_(1:T). The following describes a hybrid approach, wheremessages derived from the generative graphical model are combined withlearned messages, in short:

1. Prior Knowledge Messages: these messages may be derived from thegenerative graphical model (e.g., equations of motion from a physicsmodel).

2. Learned Messages: These messages may be learned using a Graph NeuralNetwork which may be trained to reduce the inference error on labelleddata in combination with the Prior Knowledge Messages.

Hidden Markov Model

FIG. 2 illustrates, as background, a Hidden Markov Model, in which a setof unobservable variables x_(t) may define the state of a process atevery timestep 0<t<T. The set of observable variables from which one maywant to infer the process states are denoted by y_(t). One may expressp(x_(1:T)|y_(1:T)) as the probability distribution of the hidden statesgiven the observations. It may be desirable to find which statesx_(1:T), maximize this probability distribution. More formally:

$\begin{matrix}{{\hat{x}}_{1:T} = {\underset{x_{1:T}}{argmax}{p\left( x_{1:T} \middle| y_{1:T} \right)}}} & (1)\end{matrix}$

Under the Markov assumption, these variables may follow a graphicalmodel structure where i) the transition model may be described by thetransition probability p(x_(t)|x_(t−1)), and ii) the measurement modelmay be described by p(y_(t)|x_(t)). Both distributions may be stationaryfor all t. The resulting graphical model may be be expressed with thefollowing equation:

$\begin{matrix}{{p\left( {x_{1{\vdots T}},y_{1{\vdots T}}} \right)} = {{p\left( x_{0} \right)}{\prod\limits_{t = 1}^{T}{{p\left( {x_{t}{❘x_{t - 1}}} \right)}{p\left( {y_{t}{❘x_{t}}} \right)}}}}} & (2)\end{matrix}$

A conventional approach for inference problems in this graphical modelis the Kalman Filter and Smoother. In Kalman Filters, both transitionand measurement distributions are assumed to be linear and Gaussian: theprior knowledge one may have about the process may be encoded in lineartransitions and measurement processes, and the uncertainty of thepredictions with respect to the real system may be modelled by Gaussiannoise:x _(t) =Fx _(t−1) +q _(t)  (3)y _(t) =Hx _(t) +r _(t)  (4)

Here q_(t),r_(t) may come from Gaussian distributions q_(t)˜

(0,Q), r_(t)˜

(0,R), with F, H being the linear transition and measurement functionsrespectively. If the process from which one may be inferring x_(1:T) isactually Gaussian and linear, a Kalman Filter+Smoother with the rightparameters (F,H,Q,R) will be able to infer the optimal state estimates.However, the real world is usually non-linear and complex, so assumingthat a process is linear may be a strong limitation.

To model the complexities of the real world, these complexities may belearned from data (also referred to as ‘ground truth’ or ‘trainingdata’) through learnable models such as neural networks. The followinghybrid approach, which is also referred to as a Graphical RecurrentInference Network (GRIN), combines knowledge from a generative model(e.g., physics equations) with a correction that is learned fromtraining data using a neural network. Experiments have shown that thishybrid approach outperforms the prior knowledge-based methods and alsothe neural network methods for low and high data regimes, respectively.In other words, the hybrid approach benefits from the inductive bias inthe limit of small data and also the high capacity of a neural networksin the limit of large data. The hybrid approach may gracefullyinterpolate between these different regimes

Prior Knowledge Messages

In order to define the prior knowledge messages, the approach of Putzky& Welling (https://arxiv.orq/abs/1706.04008) may be extended to theprobabilistic graphical modelling framework from equation (2). This maybe interpreted as an iterative optimization process to estimate themaximum likelihood values of the states x_(1:T). The recursive updatefor each consecutive estimate of x_(1:T) may be given by:

$\begin{matrix}{x_{1:T}^{({i + 1})} = {x_{1:T}^{(i)} + {\gamma{\nabla_{x_{1:T}^{(i)}}\log}\left( {p\left( {x_{1:T}^{(i)},y_{1:T}} \right)} \right)}}} & (5)\end{matrix}$

Simplifying equation (5) for the hidden Markov process from equation (2)may yield three input messages for each inferred node x_(t):

$\begin{matrix}{x_{t}^{({i + 1})} = {x_{t}^{(i)} + {\gamma Mp_{t}^{(i)}}}} & (6)\end{matrix}$ $\begin{matrix}{{Mp}_{t}^{(i)} = {m_{x_{t - 1}\rightarrow x_{t}}^{(i)} + m_{x_{t + 1}\rightarrow x_{t}}^{(i)} + m_{y_{t}\rightarrow x_{t}}^{(i)}}} & \end{matrix}$ $\begin{matrix}{m_{x_{t - 1}\rightarrow x_{t}}^{(i)} = {{\frac{\partial}{\partial x_{t}^{(i)}}\log}\left( {p\left( x_{t}^{(i)} \middle| x_{t - 1}^{(i)} \right)} \right)}} & (7)\end{matrix}$ $\begin{matrix}{m_{x_{t + 1}\rightarrow x_{t}}^{(i)} = {{\frac{\partial}{\partial x_{t}^{(i)}}\log}\left( {p\left( x_{t + 1}^{(i)} \middle| x_{t}^{(i)} \right)} \right)}} & (8)\end{matrix}$ $\begin{matrix}{m_{y_{t}\rightarrow x_{t}}^{(i)} = {{\frac{\partial}{\partial x_{t}^{(i)}}\log}\left( {p\left( y_{t} \middle| x_{t}^{(i)} \right)} \right)}} & (9)\end{matrix}$

The three messages may be obtained by computing the three derivativesfrom equations (7), (8), (9). It is often assumed that the transitionand measurement distributions p(x_(t)|x_(t−1)), p(y_(t)|x_(t)) arelinear and Gaussian, resulting in the Kalman Filter model. Next, theexpressions for the prior knowledge messages may be provided whenassuming these linear and Gaussian functions as in (3), (4):m _(x) _(t−1) _(→x) _(t) =−Q ⁻¹(x _(t) −Fx _(t−1))  (10)m _(x) _(t+1) _(→x) _(t) =F ^(T) Q ⁻¹(x _(t+1) −Fx _(t))  (11)m _(y) _(t) _(→x) _(t) =H ^(T) R ⁻¹(y _(t) −H _(x) _(t) )  (12)

Adding Learned Messages

This section describes turning the model into a hybrid model oralgorithm by adding a learned signal e_(t) ^((i)) to the recursiveoperation, with FIG. 4 illustrating an inference iteration of the hybridmodel. The resulting equation is:x _(t) ^((i)) =x _(t) ^((i−1))+γ(Mp _(t) ^((i−1)) +e _(t) ^((i)))  (13)

The signal e_(t) ^((i)) may be structured as graph neural network (GNN)that aggregates information between nodes. For example, a gatedrecurrent unit (GRU) may be added to the message passing operation tomake it recursive:Ml _(t) ^((i−1))=ƒ_(θ) ₁ (h _(t) ^((i−1)) , {h _(j) ^((i−1))}_(j∈)

_(t) ₂ )h _(t) ^((i)) =GRU _(θ) ₃ ([Mp_(a) _(t) ^((i−1)) , Ml _(t) ^((i−1)) ,y_(t)],h _(t) ^((i−1)))e _(t) ^((i))=ƒ_(θ) ₂ (h _(t) ^((i)))  (14)

For each time-step t one may keep track of a hidden state h_(t) ^((i))that represents a node of a directed chain graph. Two nodes (h_(t)^((i)),h_(t+1)) may be connected if they are consecutive in the temporaldimension t. The function ƒ_(θ) ₁ (·) may for example be a 2-layersGraph Neural Network that for each node, aggregates the neighborhoodinformation into the message Ml_(t) ^((i)). The GRU may receive at theinput gate the message Ml_(t) ^((i)), the concatenation of priorknowledge messages (7), (8), (9) that one may denote as Mp_(at) ^((i−1))and the observations y_(t). From this inputted data, the GRU_(θ) ₃ mayupdate the hidden state h_(t) ^((i)). Finally, an MLP ƒ_(θ) ₂ (·) maymap the hidden state h_(t) ^((i)) to the correction signal e_(t) ^((i)).

Equation (13) may thus represent a hybrid model or algorithm (alsoreferred to as a GRIN (Graphical Recurrent Inference Network) model) ina simple recursive form where x_(t) may be updated through twocontributions: one that relies on previously known equations Mp_(t)^((i)) and another one, e_(t) ^((i)), that is learned.

During training of the learned model, the loss function may be the MSEbetween the inferred states x_(1:T), and the ground truth statesgt_(1:T). In order to provide early feedback, the loss function may becomputed at every iteration with a weighted sum that emphasizes lateriterations,

${w_{i} = \frac{i}{N}},$more formally:

$\begin{matrix}{{{Loss}(\Theta)} = {\sum\limits_{i = 1}^{N}{w_{i}{{MSE}\left( {{gt}_{1:T} - x_{1:T}^{(i)}} \right)}}}} & (15)\end{matrix}$

In some examples, the ground truth may only contain part of the inferredstate X_(1:T). For example, in some localization tasks the state maydescribe the position and velocity of a process while only the groundtruth may be available for the position. In such situations, the lossmay be computed with the part of the state that is comprised in theground truth. The resulting loss may be the following:

$\begin{matrix}{{{Loss}(\Theta)} = {\sum\limits_{i = 1}^{N}{w_{i}{{MSE}\left( {{gt}_{1:T} - {Bx}_{1:T}^{(i)}} \right)}}}} & (16)\end{matrix}$where B is a matrix whose rows are one-hot encoded vectors that selectthe components from x_(1:T) contained in the ground truth.

The training of the learned model may comprise three main steps. First,each x_(t) ⁽⁰⁾ may be initialized to an initial value. In order to speedup the convergence, a value may be chosen that maximizes p(y_(t)|x_(t)).For example, in a trajectory estimation context, the position values ofx_(t) may be set to the observed positions y_(t). Second, thehyperparameters of the prior knowledge model may be tuned as it would bedone with a Kalman Filter, which are usually the variance of themeasurement and transition Gaussian distributions. Finally, the learnedmodel may be trained using the above-mentioned loss function (15), (16).

In three different datasets for trajectory estimation, namely a linearsynthetic dataset, a non-linear chaotic system (Lorenz attractor) and areal-world positioning system (Michigan NCLT dataset), the GRIN modelhas been found to efficiently combine prior knowledge messages withlearned ones, outperforming either learned or graphical inference inisolation for different data regimes.

FIG. 4 shows a computer-implemented method 200 for enabling control ofphysical system based on a state of the physical system which isinferred from sensor data. The method 200 may correspond to an operationof the system as described with reference to FIG. 1 and elsewhere.However, this is not a limitation, in that the method may also beimplemented by another system, apparatus or device.

The method 200 may comprise, in an operation titled “ACCESSING SENSORDATA”, using an input interface, accessing 210 sensor data representingsensor measurements associated with the state of the physical system.The method 200 may further comprise iteratively inferring the state ofthe physical system based on the sensor data by, in an iteration 220 ofthe iterative inference, in an operation titled “OBTAINING SENSORMEASUREMENT” obtaining 230 a sensor measurement from the sensor data, inan operation titled “OBTAINING INITIAL INFERENCE FROM MATHEMATICALMODEL” obtaining 240 an initial inference of the state using amathematical model representing a prior knowledge-based modelling of thestate as a function of the sensor measurement and a previous inferredstate, in an operation titled “OBTAINING CORRECTION VALUE FROM LEARNEDMODEL” applying 250 a learned model to the initial inference of thestate and the sensor measurement, wherein the learned model has beenlearned to minimize an error between initial inferences provided by themathematical model and a ground truth and to provide a correction valueas output for correcting the initial inference of the state of themathematical model, and in an operation titled “COMBINING INITIALINFERENCE WITH CORRECTION VALUE” obtaining 260 a current inferred stateby combining the initial inference of the state with the correctionvalue. The method 200 may further comprise, in an operation titled“PROVIDING OUTPUT DATA TO OUTPUT DEVICE”, using an output interface,providing 270 output data to an output device which is used in thecontrol of the physical system to enable the control of the physicalsystem based on the current inferred state. It will be appreciated that,in general, the operations of method 200 of FIG. 4 may be performed inany suitable order, e.g., consecutively, simultaneously, or acombination thereof, subject to, where applicable, a particular orderbeing necessitated, e.g., by input/output relations between therespective operations.

The method may be implemented on a computer as a computer implementedmethod, as dedicated hardware, or as a combination of both. As alsoillustrated in FIG. 5 , instructions for the computer, e.g., executablecode, may be stored on a computer readable medium 300, e.g., in the formof a series 310 of machine-readable physical marks and/or as a series ofelements having different electrical, e.g., magnetic, or opticalproperties or values. The executable code may be stored in a transitoryor non-transitory manner. Examples of computer readable mediums includememory devices, optical storage devices, integrated circuits, servers,online software, etc. FIG. 5 shows an optical disc 300. Alternatively,the computer readable medium 300 may comprise transitory ornon-transitory data 310 representing a learned model as describedelsewhere in this specification.

Examples, embodiments or optional features, whether indicated asnon-limiting or not, are not to be understood as limiting the presentinvention.

It should be noted that the above-mentioned embodiments illustraterather than limit the present invention, and that those skilled in theart will be able to design many alternative embodiments withoutdeparting from the scope of the present invention. Use of the verb“comprise” and its conjugations does not exclude the presence ofelements or stages other than those stated. The article “a” or “an”preceding an element does not exclude the presence of a plurality ofsuch elements. Expressions such as “at least one of” when preceding alist or group of elements represent a selection of all or of any subsetof elements from the list or group. For example, the expression, “atleast one of A, B, and C” should be understood as including only A, onlyB, only C, both A and B, both A and C, both B and C, or all of A, B, andC. The present invention may be implemented by hardware comprisingseveral distinct elements, and by a suitably programmed computer. Whenmeans are enumerated, several of these means may be embodied by one andthe same item of hardware. The mere fact that certain measures areseparately described does not indicate that a combination of thesemeasures cannot be used to advantage.

What is claimed is:
 1. A physical system comprising: a sensor; anactuator; and processing hardware; wherein: operation of the actuator iscontrolled by the processing hardware based on a state of the physicalsystem which is inferred from sensor data; the processing hardwareincludes: an input interface configured for accessing the sensor datafrom the sensor and representing sensor measurements associated with thestate of the physical system; an output interface to the actuator viawhich the processing hardware performs the control of the operation ofthe actuator; and a processor subsystem configured to use the sensordata by: obtaining a sensor measurement from the sensor data accessedvia the input interface; applying the sensor measurement obtained fromthe accessed sensor data to a mathematical prior knowledge-based statemodel that identifies an initial inference of the state of the physicalsystem that corresponds to the sensor measurement; applying the initialinference of the state of the physical system identified by the modeland the sensor measurement obtained from the accessed sensor data to arecurrent neural network that the processor subsystem executed and thathas been learned to minimize an error between initial inferencesidentified by the mathematical prior knowledge-based state model and aground truth and to provide a final inference of the state of thephysical system; and performing the control of the operation of theactuator based on the final inference of the state of the physicalsystem; the execution, by the processor subsystem, of the recurrentneural network, to which the initial inference of the state of thephysical system and the sensor measurement has been applied, includesperforming an iterative refinement of the inference gradually from theinitial inference of the state of the physical system to the finalinference of the state of the physical system, the iterative refinementincluding performing a plurality of iterations that include: a firstiteration that operates on the sensor measurement applied to themathematical prior knowledge-based model and on the initial inference ofthe state of the physical system obtained from the mathematical priorknowledge-based model; a final iteration; and at least one intermediateiteration between the first iteration and the final iteration; each ofthe at least one intermediate iteration obtains as input, and processes,the sensor measurement that had been applied to the mathematical priorknowledge-based model, a respective modified inference of the state ofthe physical system that had been output by an immediately preceding oneof the iterations, and a respective hidden state of the recurrentnetwork that had been output by the immediately preceding one of theiterations, the processing by the respective intermediate iterationproducing: a further hidden state; and a further modified inference ofthe state of the physical system by generating a correction value as afunction of the input sensor measurement, the input respective modifiedinference of the state of the physical system, and the input respectivehidden state and then modifying the input respective modified inferenceof the state of the physical system by the correction value; and thefinal iteration obtains as input, and processes, the sensor measurementthat had been applied to the mathematical prior knowledge-based model,the further modified inference of the state of the physical system thathad been produced by an immediately preceding one of the at least oneintermediate iteration, and the further hidden state that had beenproduced by the immediately preceding one of the at least oneintermediate iteration, the processing by the final iteration producingthe final inference of the state of the physical system by generating afurther correction value as a function of the input sensor measurement,the input further modified inference of the state of the physicalsystem, and the further hidden state that had been produced by theimmediately preceding one of the at least one intermediate iteration,and then modifying the input further modified inference of the state ofthe physical system by the further correction value.
 2. The systemaccording to claim 1, wherein the processor subsystem is configured to:using the input interface, obtain a time-series of sensor measurements;obtain the initial inference of the state by using the time-series ofsensor measurements and a time-series of previous inferred states asinput to the mathematical prior knowledge-based state model; apply therecurrent neural network to the initial inference of the state and thetime-series of sensor measurements to obtain a time-series of correctionvalues; and obtain the final inference of the state by combining theinitial inference of the state with the time-series of correctionvalues.
 3. The system according to claim 1, wherein the recurrent neuralnetwork includes a gated recurrent unit to establish recursion in therecurrent neural network.
 4. The system according to claim 1, whereinthe mathematical prior knowledge-based state model provides aphysics-based modelling of the state as a function of the sensormeasurement and a previous inferred state.
 5. The system according toclaim 1, wherein the mathematical prior knowledge-based state modelincludes a transitional model part which models a conditionalprobability of the state to be inferred given a previous inferred state,and a measurement model part which models a conditional probability ofthe sensor measurement given the state to be inferred.
 6. The systemaccording to claim 5, wherein the processor subsystem is configured toiteratively infer the state by Kalman Filtering by assuming thatprobability distributions of the mathematical prior knowledge-basedstate model are linear and Gaussian.
 7. The system according to claim 1,wherein the actuator is a vehicle actuator or a robotics actuator. 8.The system according to claim 1, wherein the system is configured tocause a sensory perceptible warning signal to be generated when thefinal inference of the state of the physical system indicates a failureof the physical system.
 9. A method of a physical system that includes asensor, an actuator, and processing hardware in which operation of theactuator is controlled by the processing hardware based on a state ofthe physical system which is inferred from sensor data, the methodcomprising the following steps: using an input interface of theprocessing hardware, accessing the sensor data from the sensor andrepresenting sensor measurements associated with the state of thephysical system; performing the following, by a processor subsystem ofthe processing hardware, using the sensor data: obtaining a sensormeasurement from the sensor data from the sensor accessed via the inputinterface; applying the sensor measurement obtained from the accessedsensor data to a mathematical prior knowledge-based model thatidentifies an initial inference of the state of the physical system thatcorresponds to the sensor measurement; applying the initial inference ofthe state of the physical system identified by the model and the sensormeasurement obtained from the accessed sensor data to a recurrent neuralnetwork that is executed by the processor subsystem and that has beenlearned to minimize an error between initial inferences identified bythe mathematical prior knowledge-based state model and a ground truthand to provide a final inference of the state of the physical system;and performing the control of the operation of the actuator based on thefinal inference of the physical system; wherein: the execution, by theprocessor system of the recurrent neural network, to which the initialinference of the state of the physical system and the sensor measurementhas been applied, includes performing an iterative refinement of theinference gradually from the initial inference of the state of thephysical system to the final inference of the state of the physicalsystem, the iterative refinement including performing a plurality ofiterations that include: a first iteration that operates on the sensormeasurement applied to the mathematical prior knowledge-based model andon the initial inference of the state of the physical system obtainedfrom the mathematical prior knowledge-based model; a final iteration;and at least one intermediate iteration between the first iteration andthe final iteration; each of the at least one intermediate iterationobtains as input, and processes, the sensor measurement that had beenapplied to the mathematical prior knowledge-based model, a respectivemodified inference of the state of the physical system that had beenoutput by an immediately preceding one of the iterations, and arespective hidden state of the recurrent neural network that had beenoutput by the immediately preceding one of the iterations, theprocessing by the respective intermediate iteration producing: a furtherhidden state; and a further modified inference of the state of thephysical system by generating a correction value as a function of theinput sensor measurement, the input respective modified inference of thestate of the physical system, and the input respective hidden state andthen modifying the input respective modified inference of the state ofthe physical system by the correction value; and the final iterationobtains as input, and processes, the sensor measurement that had beenapplied to the mathematical prior knowledge-based model, the furthermodified inference of the state of the physical system that had beenproduced by an immediately preceding one of the at least oneintermediate iteration, and the further hidden state that had beenproduced by the immediately preceding one of the at least oneintermediate iteration, the processing by the final iteration producingthe final inference of the state of the physical system by generating afurther correction value as a function of the input sensor measurement,the input further modified inference of the state of the physicalsystem, and the further hidden state that had been produced by theimmediately preceding one of the at least one intermediate iteration,and then modifying the input further modified inference of the state ofthe physical system by the further correction value.